Skip to content

Fix: Cast exception occurs if function/source/sink type is ByteBuffer#11611

Merged
jerrypeng merged 1 commit into
apache:masterfrom
jerrypeng:fix_pulsar_functions_byte_buffer
Aug 10, 2021
Merged

Fix: Cast exception occurs if function/source/sink type is ByteBuffer#11611
jerrypeng merged 1 commit into
apache:masterfrom
jerrypeng:fix_pulsar_functions_byte_buffer

Conversation

@jerrypeng

@jerrypeng jerrypeng commented Aug 10, 2021

Copy link
Copy Markdown
Contributor

Motivation

If a Pulsar Function/Source/Sink is of type ByteBuffer, an cast exception will occur:

6:44:00.226 INFO  [external-repl-prop/io/PulsarSink-test-0] o.a.p.f.i.JavaInstanceRunnable@357 - Encountered exception in sink write: 
java.lang.ClassCastException: class [B cannot be cast to class java.nio.ByteBuffer ([B and java.nio.ByteBuffer are in module java.base of loader 'bootstrap')
	at org.apache.pulsar.functions.worker.PulsarFunctionLocalRunTest$StatsNullSink.write(PulsarFunctionLocalRunTest.java:1027) ~[test-classes/:?]
	at org.apache.pulsar.functions.instance.JavaInstanceRunnable.sendOutputMessage(JavaInstanceRunnable.java:355) [classes/:?]
	at org.apache.pulsar.functions.instance.JavaInstanceRunnable.handleResult(JavaInstanceRunnable.java:338) [classes/:?]
	at org.apache.pulsar.functions.instance.JavaInstanceRunnable.run(JavaInstanceRunnable.java:283) [classes/:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]

The is because if the function is of type ByteBuffer, the schema to used in the consumer is BYTES, which is not correct

Modifications

Use the correct ByteBuffer schema when the type of the function/source/sink is ByteBuffer

Verifying this change

Added a test

Documentation

No changes necessary

For committer

For this PR, do we need to update docs?

No

@jerrypeng jerrypeng added type/bug The PR fixed a bug or issue reported a bug area/function area/connector labels Aug 10, 2021
@jerrypeng jerrypeng added this to the 2.9.0 milestone Aug 10, 2021
@jerrypeng jerrypeng self-assigned this Aug 10, 2021
@jerrypeng
jerrypeng force-pushed the fix_pulsar_functions_byte_buffer branch from 1c1464e to 766381f Compare August 10, 2021 00:04
@jerrypeng
jerrypeng merged commit b7e027b into apache:master Aug 10, 2021
hangc0276 pushed a commit that referenced this pull request Aug 12, 2021
@jerrypeng Jerry Peng @hangc0276
Fix: Cast exception occurs if function/source/sink type is ByteBuffer (…
33c9fb7 
…#11611)

Co-authored-by: Jerry Peng <jerryp@splunk.com>
(cherry picked from commit b7e027b)
@hangc0276 hangc0276 added the cherry-picked/branch-2.8 Archived: 2.8 is end of life label Aug 12, 2021
jerrypeng added a commit to jerrypeng/incubator-pulsar that referenced this pull request Aug 18, 2021
@jerrypeng Jerry Peng
Fix: Cast exception occurs if function/source/sink type is ByteBuffer (…
c3a5b8b 
…apache#11611)

Co-authored-by: Jerry Peng <jerryp@splunk.com>
jerrypeng added a commit to jerrypeng/incubator-pulsar that referenced this pull request Nov 4, 2021
@jerrypeng Jerry Peng
Fix: Cast exception occurs if function/source/sink type is ByteBuffer (…
b0b7465 
…apache#11611)

Co-authored-by: Jerry Peng <jerryp@splunk.com>
dlg99 pushed a commit to dlg99/pulsar that referenced this pull request Mar 4, 2022
@jerrypeng Jerry Peng @dlg99
Fix: Cast exception occurs if function/source/sink type is ByteBuffer (…
fb8e6e9 
…apache#11611)

Co-authored-by: Jerry Peng <jerryp@splunk.com>
(cherry picked from commit b7e027b)
eolivelli pushed a commit to datastax/pulsar that referenced this pull request Mar 7, 2022
@jerrypeng Jerry Peng @eolivelli
Fix: Cast exception occurs if function/source/sink type is ByteBuffer (…
9b23e26 
…apache#11611)

Co-authored-by: Jerry Peng <jerryp@splunk.com>
(cherry picked from commit b7e027b)
bharanic-dev pushed a commit to bharanic-dev/pulsar that referenced this pull request Mar 18, 2022
@jerrypeng Jerry Peng @splunk-xingw
Fix: Cast exception occurs if function/source/sink type is ByteBuffer (…
4b896c1 
…apache#11611)

Co-authored-by: Jerry Peng <jerryp@splunk.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@merlimat merlimat merlimat approved these changes
@cckellogg cckellogg cckellogg approved these changes
@srkukarni srkukarni Awaiting requested review from srkukarni
@aahmed-se aahmed-se Awaiting requested review from aahmed-se

Assignees

@jerrypeng jerrypeng

Labels

area/connector area/function cherry-picked/branch-2.8 Archived: 2.8 is end of life release/2.8.1 type/bug The PR fixed a bug or issue reported a bug

Projects

None yet

Milestone

2.9.0

Development

Successfully merging this pull request may close these issues.

4 participants

@jerrypeng @merlimat @cckellogg @hangc0276